Skip to content

Conversation

@jpshackelford
Copy link
Contributor

@jpshackelford jpshackelford commented Jan 8, 2026

Summary

Implements #1645 - adds the ability to fetch plugins from remote sources (GitHub repositories, git URLs) and cache them locally. This is a building block for the Plugin Directory feature (OpenHands/OpenHands#12088).

Changes

New Function: Plugin.fetch()

@classmethod
def fetch(
    cls,
    source: str,
    cache_dir: Path | None = None,
    ref: str | None = None,
    update: bool = True,
) -> Path:
    """Fetch a plugin from a remote source and return the local cached path."""

Source Parsing

The parse_plugin_source() function handles various source formats:

  • "github:owner/repo" - GitHub repository shorthand
  • "https://github.com/owner/repo.git" - Full git URL
  • "[email protected]:owner/repo.git" - SSH git URL
  • "/local/path" - Local path (returned as-is)

Caching Strategy

  • Cache location: ~/.openhands/cache/plugins/{name}-{hash}/
  • If cached and exists: optionally update (git fetch) or use as-is
  • If not cached: clone the repository
  • Supports shallow clones for efficiency (--depth 1)

Usage Example

# Fetch and load a plugin
path = Plugin.fetch("github:owner/my-plugin")
plugin = Plugin.load(path)

# With specific version
path = Plugin.fetch("github:owner/my-plugin", ref="v1.0.0")
plugin = Plugin.load(path)

# Fetch and load in one step
plugin = Plugin.load(Plugin.fetch("github:owner/my-plugin"))

Files Changed

  • openhands-sdk/openhands/sdk/plugin/fetch.py - New module with fetching logic
  • openhands-sdk/openhands/sdk/plugin/plugin.py - Added fetch() classmethod
  • openhands-sdk/openhands/sdk/plugin/__init__.py - Export new types
  • tests/sdk/plugin/test_plugin_fetch.py - 34 new unit tests

Checklist

  • Plugin.fetch("github:owner/repo") clones and returns local path
  • Plugin.fetch("https://...") works with any git URL
  • Plugin.fetch("/local/path") returns path unchanged
  • Caching works - second fetch uses cached version
  • ref parameter checks out specific branch/tag
  • Errors are handled gracefully with PluginFetchError
  • Unit tests cover all source types and caching behavior (34 tests)

Related Issues

@jpshackelford can click here to continue refining the PR


Agent Server images for this PR

GHCR package: https://github.com/OpenHands/agent-sdk/pkgs/container/agent-server

Variants & Base Images

Variant Architectures Base Image Docs / Tags
java amd64, arm64 eclipse-temurin:17-jdk Link
python amd64, arm64 nikolaik/python-nodejs:python3.12-nodejs22 Link
golang amd64, arm64 golang:1.21-bookworm Link

Pull (multi-arch manifest)

# Each variant is a multi-arch manifest supporting both amd64 and arm64
docker pull ghcr.io/openhands/agent-server:6107e20-python

Run

docker run -it --rm \
  -p 8000:8000 \
  --name agent-server-6107e20-python \
  ghcr.io/openhands/agent-server:6107e20-python

All tags pushed for this build

ghcr.io/openhands/agent-server:6107e20-golang-amd64
ghcr.io/openhands/agent-server:6107e20-golang_tag_1.21-bookworm-amd64
ghcr.io/openhands/agent-server:6107e20-golang-arm64
ghcr.io/openhands/agent-server:6107e20-golang_tag_1.21-bookworm-arm64
ghcr.io/openhands/agent-server:6107e20-java-amd64
ghcr.io/openhands/agent-server:6107e20-eclipse-temurin_tag_17-jdk-amd64
ghcr.io/openhands/agent-server:6107e20-java-arm64
ghcr.io/openhands/agent-server:6107e20-eclipse-temurin_tag_17-jdk-arm64
ghcr.io/openhands/agent-server:6107e20-python-amd64
ghcr.io/openhands/agent-server:6107e20-nikolaik_s_python-nodejs_tag_python3.12-nodejs22-amd64
ghcr.io/openhands/agent-server:6107e20-python-arm64
ghcr.io/openhands/agent-server:6107e20-nikolaik_s_python-nodejs_tag_python3.12-nodejs22-arm64
ghcr.io/openhands/agent-server:6107e20-golang
ghcr.io/openhands/agent-server:6107e20-java
ghcr.io/openhands/agent-server:6107e20-python

About Multi-Architecture Support

  • Each variant tag (e.g., 6107e20-python) is a multi-arch manifest supporting both amd64 and arm64
  • Docker automatically pulls the correct architecture for your platform
  • Individual architecture tags (e.g., 6107e20-python-amd64) are also available if needed

Implements #1645 - adds the ability to fetch plugins from remote sources
(GitHub repositories, git URLs) and cache them locally.

Changes:
- Add Plugin.fetch() classmethod to fetch from remote sources
- Add parse_plugin_source() to parse various source formats:
  - GitHub shorthand: 'github:owner/repo'
  - Git URLs: HTTPS, SSH, git:// protocol
  - Local paths (returned as-is)
- Add PluginFetchError exception for fetch failures
- Implement caching at ~/.openhands/cache/plugins/
- Support shallow clones for efficiency
- Support specific ref (branch/tag/commit) checkout
- Add comprehensive unit tests (34 new tests)

Co-authored-by: openhands <[email protected]>
@github-actions
Copy link
Contributor

github-actions bot commented Jan 8, 2026

Coverage

Coverage Report •
FileStmtsMissCoverMissing
openhands/sdk/context/skills
   utils.py1431887%44, 61, 99, 103, 130–131, 134, 149–150, 202, 214, 235, 256, 306–308, 378–379
openhands/sdk/git
   cached_repo.py79494%181, 191–193
   utils.py872670%71–73, 98–100, 134–135, 142–147, 152–153, 163–168, 178–180, 208
openhands/sdk/plugin
   plugin.py1552285%274–276, 278–283, 299–302, 311–315, 331–332, 350–351
TOTAL15937489269% 

- Add blank line after imports in fetch.py
- Remove unused import PluginFetchError in plugin.py
- Reformat long lines in test file

Co-authored-by: openhands <[email protected]>
- Create GitHelper class to encapsulate all git operations (clone, fetch, checkout, etc.)
- Refactor fetch.py to use GitHelper via dependency injection
- Update unit tests to mock GitHelper instead of subprocess.run
- Add integration tests for real git operations
- Add support for file:// URLs for local testing

This improves test coverage by:
1. Allowing unit tests to execute actual fetch.py logic while mocking git operations
2. Providing separate integration tests that use real git operations

Co-authored-by: openhands <[email protected]>
openhands-agent and others added 6 commits January 8, 2026 21:36
Add comprehensive tests to improve coverage for the plugin module:

- fetch.py: Add tests for set_git_helper(), relative path parsing,
  default cache_dir fallback, and PluginFetchError re-raise
- git_helper.py: Add unit tests for error handling paths including
  CalledProcessError and TimeoutExpired for all git operations
- plugin.py: Add tests for plugin loading edge cases including
  manifest parsing, skills/agents/commands loading, and error handling
- types.py: Add tests for AgentDefinition and CommandDefinition
  frontmatter parsing including complex field handling

Simplify integration test to avoid skill loading complexity.

Co-authored-by: openhands <[email protected]>
The --forked flag was preventing coverage from being collected properly
because pytest-forked uses os.fork() which doesn't combine coverage data.

Changes:
- Split SDK test run: forked tests run separately, then non-forked tests
  with --cov-append to combine coverage data
- Add 'forked' pytest marker to identify tests needing process isolation
- Add coverage configuration with parallel mode and branch coverage
- Add coverage report exclusions for common non-coverable patterns

This allows most tests to run without forking for accurate coverage,
while still supporting tests that need process isolation.

Co-authored-by: openhands <[email protected]>
The change to run tests without --forked exposed pre-existing test pollution
issues in test_state_serialization.py where tests share state incorrectly.

Reverting to the original --forked approach until those tests are fixed.
Coverage collection will remain affected by --forked, but tests will pass.

Co-authored-by: openhands <[email protected]>
Copy link
Contributor Author

Requested Changes: Add subpath parameter

Context

During testing, we identified a bug where plugin paths (e.g., github:owner/repo/plugins/sub-plugin) were being rejected by parse_plugin_source() because it validates that GitHub shorthand has only one / in the repo path. Rather than embedding the path in the source string, we've decided to keep path as a separate parameter throughout the entire stack for consistency and clarity.

Changes Needed

openhands-sdk/openhands/sdk/plugin/fetch.py - Add subpath parameter to fetch_plugin():

def fetch_plugin(
    source: str,
    cache_dir: Path | None = None,
    ref: str | None = None,
    update: bool = True,
    subpath: str | None = None,  # NEW PARAMETER
    git_helper: GitHelper | None = None,
) -> Path:
    """Fetch a plugin from a remote source and return the local cached path.

    Args:
        source: Plugin source - "github:owner/repo", git URL, or local path
        cache_dir: Directory for caching
        ref: Optional branch, tag, or commit
        update: If True, update cached repo
        subpath: Optional subdirectory path within the repo
        git_helper: GitHelper instance

    Returns:
        Path to the plugin directory (with subpath applied if specified)
    """
    source_type, url = parse_plugin_source(source)
    
    # ... existing clone/update logic ...
    
    # NEW: Apply subpath if specified (add before final return)
    if subpath:
        final_path = plugin_path / subpath.strip('/')
        if not final_path.exists():
            raise PluginFetchError(
                f"Subdirectory '{subpath}' not found in plugin repository"
            )
        return final_path
    
    return plugin_path

openhands-sdk/openhands/sdk/plugin/plugin.py - Add subpath parameter to Plugin.fetch():

@classmethod
def fetch(
    cls,
    source: str,
    cache_dir: Path | None = None,
    ref: str | None = None,
    update: bool = True,
    subpath: str | None = None,  # NEW PARAMETER
) -> Path:
    """Fetch a plugin from a remote source."""
    return fetch_plugin(
        source, 
        cache_dir=cache_dir, 
        ref=ref, 
        update=update,
        subpath=subpath,  # NEW
    )

This maintains API shape consistency across all layers (see OpenHands/OpenHands#12087 (comment) for the full API design).

jpshackelford pushed a commit that referenced this pull request Jan 12, 2026
Add plugin_path field to StartConversationRequest and pass it as
subpath parameter to Plugin.fetch(). This enables fetching plugins
from subdirectories within repositories (e.g., monorepos with
multiple plugins).

Changes:
- models.py: Add plugin_path optional field to StartConversationRequest
- conversation_service.py: Pass plugin_path as subpath to Plugin.fetch()
- test_conversation_service.py: Update tests to verify subpath handling

Note: This depends on PR #1647 adding the subpath parameter to Plugin.fetch()

Co-authored-by: openhands <[email protected]>
Add optional subpath parameter to both fetch_plugin() and Plugin.fetch()
methods. This allows fetching plugins from subdirectories within a
repository, which is needed when plugin paths are specified separately
rather than embedded in the source string.

Changes:
- Add subpath parameter to fetch_plugin() in fetch.py
- Add subpath parameter to Plugin.fetch() in plugin.py
- Apply subpath to both local paths and remote repositories
- Strip leading/trailing slashes from subpath
- Raise PluginFetchError if subpath doesn't exist
- Add 8 new unit tests for subpath functionality

Addresses review comment on PR #1647.

Co-authored-by: openhands <[email protected]>
jpshackelford pushed a commit that referenced this pull request Jan 12, 2026
Add plugin_path field to StartConversationRequest and pass it as
subpath parameter to Plugin.fetch(). This enables fetching plugins
from subdirectories within repositories (e.g., monorepos with
multiple plugins).

Changes:
- models.py: Add plugin_path optional field to StartConversationRequest
- conversation_service.py: Pass plugin_path as subpath to Plugin.fetch()
- test_conversation_service.py: Update tests to verify subpath handling

Note: This depends on PR #1647 adding the subpath parameter to Plugin.fetch()

Co-authored-by: openhands <[email protected]>
@enyst
Copy link
Collaborator

enyst commented Jan 12, 2026

@OpenHands Install yourself gh if you don't already have it, and then get all this PR diff and the inline comments to it.

GOAL: answer a code design question: does this feature implementation belong in the sdk package of the SDK (like this PR), in the workspace package of the SDK (Workspace classes), or in the client apps?

I want you to make a throughout research in the SDK packages and try to understand the git-related operations and responsibilities. Answer directly in a github comment.

@openhands-ai
Copy link

openhands-ai bot commented Jan 12, 2026

I'm on it! enyst can track my progress at all-hands.dev

Copy link
Collaborator

enyst commented Jan 12, 2026

Thanks for flagging this — I dug through the SDK packages to see where “git-ish” responsibilities currently live.

What exists today

SDK (openhands-sdk) already does small, self-contained git + caching for SDK-level assets.

  • openhands.sdk.context.skills.utils.update_skills_repository() maintains a local clone under ~/.openhands/cache/skills/... and does git clone/fetch/reset via subprocess.
    • This is not workspace-scoped: it’s a pure “fetch a read-only dependency into a user cache” helper.

Workspace (openhands-workspace) is about running commands / managing environments, not about bringing in additional repos into ~/.openhands caches.

  • The workspace layer exposes git introspection APIs (git_changes, git_diff) for the repo inside the workspace, but it does not implement git clone / remote fetching logic. It’s basically “operate on whatever repo is already in the workspace”.

So where should Plugin.fetch() live?

Assessment: Plugin.fetch() belongs in openhands-sdk (the sdk package), but ideally as a shared “remote repo fetch + cache” utility used by both skills + plugins, not in Workspace or client apps.

Why SDK (not Workspace):

  • The feature is “resolve a plugin source string → ensure a local cached copy exists → return a filesystem path”. That mirrors the existing public-skills behavior and is independent of any particular workspace implementation (Docker, API remote, cloud, etc).
  • The cache location is under the user’s home (~/.openhands/cache/...), which is outside the workspace’s conceptual boundary. Workspaces should generally avoid mutating host-level caches (especially for remote workspaces).
  • Workspace APIs today are about a repo that already exists in the workspace (diff/changes), not “bring external dependencies into existence”.

Why not client apps:

  • If every client re-implements “git clone with caching + ref checkout + update semantics”, we’ll end up with duplicated, inconsistent behavior. This is a reusable SDK primitive (plugins, skills registry, future org registries, etc).
  • Keeping it in the SDK makes it testable and available to multiple clients.

Design suggestion (to reduce duplication)

Right now the PR introduces openhands.sdk.plugin.git_helper.GitHelper while openhands.sdk.git.utils and skills/utils.py already contain git wrappers.

I’d recommend unifying these as:

  1. a single openhands.sdk.git abstraction for both "git introspection" and "git fetch/clone" operations, and
  2. small domain wrappers:
    • skills calls something like sdk.git.cached_clone_or_update(...)
    • plugin calls the same and adds “parse plugin source → choose cache key/name”.

That keeps git operations centralized and avoids the “we have 2-3 ways to call git” situation.

@openhands-ai
Copy link

openhands-ai bot commented Jan 12, 2026

Summary of work completed

  • Installed the GitHub CLI (gh) in the environment (it was not present).
  • Retrieved PR feat(plugin): Add Plugin.fetch() for remote plugin fetching and caching #1647 metadata and the full diff (gh pr diff 1647 …) and reviewed the implementation (Plugin.fetch + parsing + caching + new git helper).
  • Pulled all PR-level comments (gh pr view --comments) and all inline review comments via the GitHub API endpoint for pull request review comments (/pulls/1647/comments), and reviewed them (notably the concern about duplicating existing git clone/checkout logic and whether this belongs in workspace).
  • Performed a repo-wide audit of existing git responsibilities across packages:
    • Found existing SDK git operations and caching behavior in openhands-sdk, specifically openhands.sdk.context.skills.utils.update_skills_repository() which clones/fetches/resets into ~/.openhands/cache/skills.
    • Confirmed openhands-workspace focuses on workspace lifecycle + executing commands and does not implement remote clone/fetch-to-cache logic; SDK workspace exposes git introspection APIs (git_changes, git_diff) for repos already present.
    • Reviewed openhands.sdk.git.* utilities (safe git command wrapper, diff/changes helpers) to understand current patterns.
  • Posted a GitHub PR comment answering the design question directly:
    • Conclusion: Plugin.fetch() belongs in openhands-sdk, not Workspace or client apps.
    • Additional recommendation: reduce duplication by centralizing “clone/fetch/update cached repo” primitives under openhands.sdk.git and have both skills and plugin use that shared utility.

Checklist vs user request

  • Install gh if missing
  • Get PR diff
  • Get inline comments
  • Research SDK packages for git-related responsibilities
  • Answer the design question directly in a GitHub comment

Code changes / pushes

  • No repository code changes were made.
  • No commits were created.
  • Nothing to push.

View full conversation

Move git clone, fetch, checkout, and cache management into a new shared
module at openhands.sdk.git.cached_repo. This consolidates duplicate
git operations that were in both:
- openhands.sdk.context.skills.utils (update_skills_repository)
- openhands.sdk.plugin.git_helper (GitHelper class)

Changes:
- Add openhands.sdk.git.cached_repo with GitHelper class and
  cached_clone_or_update() function
- Refactor skills/utils.py to use the shared cached_clone_or_update()
- Refactor plugin/fetch.py to use shared git utilities
- Delete plugin/git_helper.py (consolidated into git/cached_repo.py)
- Update plugin/__init__.py exports to use new locations
- Update tests to mock the correct module paths

This addresses review feedback about duplicate git operations in the
codebase, centralizing them in the sdk.git package for reuse by both
skills and plugins.

Co-authored-by: openhands <[email protected]>
@openhands-ai
Copy link

openhands-ai bot commented Jan 14, 2026

Looks like there are a few issues preventing this PR from being merged!

  • GitHub Actions are failing:
    • Pre-commit checks

If you'd like me to help, just leave a comment, like

@OpenHands please fix the failing actions on PR #1647 at branch `feat/plugin-fetch`

Feel free to include any additional details that might help me get this PR into a better state.

You can manage your notification settings

Refactor GitHelper class in cached_repo.py to use the existing
run_git_command() utility from git/utils.py instead of calling
subprocess directly. This provides:

- Single code path for all git subprocess invocations
- Consistent error handling via GitCommandError
- Centralized timeout handling and logging
- Better code self-similarity across the git module

Changes:
- Add timeout parameter to run_git_command() (default: 30s)
- Refactor GitHelper methods to delegate to run_git_command()
- Update exception handling from GitError to GitCommandError
- Export GitCommandError from plugin module for backwards compatibility
- Update tests to use correct mock paths and error patterns

Co-authored-by: openhands <[email protected]>
Copy link
Contributor Author

jpshackelford commented Jan 15, 2026

@enyst:

Git Operations Consolidation - Complete

Per your feedback about duplicate git operations, we've completed a full consolidation. Here's what was done:

Summary

We've refactored GitHelper in cached_repo.py to use the existing run_git_command() utility from git/utils.py instead of calling subprocess directly. This eliminates the code duplication you identified.

Changes Made

  1. Extended run_git_command() with an optional timeout parameter (default: 30s) to support the varying timeout needs of different git operations (clone: 120s, fetch: 60s, etc.)

  2. Refactored GitHelper class - All methods now delegate to run_git_command():

    • clone(), fetch(), checkout(), reset_hard(), get_current_branch()
    • Removed ~70 lines of duplicate subprocess handling code
  3. Unified error handling - All git operations now use GitCommandError (the structured exception with command, exit_code, and stderr fields) instead of the simpler GitError with just a message string

  4. Updated exports - Added GitCommandError to the plugin module's public API for backwards compatibility

Result: Single Code Path

All git subprocess calls in the SDK now flow through run_git_command():

git/utils.py:run_git_command()     ← Single entry point for all git subprocess calls
    ↑
git/cached_repo.py:GitHelper       ← Used by skills & plugins
    ↑
├── context/skills/utils.py        ← Skills repository caching  
└── plugin/fetch.py                ← Plugin repository caching

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(plugin): Add Plugin.fetch() for remote plugin fetching and caching

4 participants